Lococode Performs Nonlinear Ica without Knowing the Number of Sources

Low-complexity coding and decoding (Lococode), a novel approach to sensory coding, trains autoassocia-tors (AAs) by Flat Minimum Search (FMS), a recent general method for nding low-complexity networks with high generalization capability. FMS works by minimizing both training error and required weight precision. We nd that as a by-product Lococode separates nonlinear superpositions of sources without knowing their number. Assuming that the input data can be reduced to few simple causes (this is often the case with visual data), according to our theoretical analysis the hidden layer of an FMS-trained AA tends to code each input by a sparse code based on as few simple, independent features as possible. In experiments Lo-cocode extracts optimal codes for diicult, nonlinear versions of the \noisy bars" benchmark problem, while traditional ICA and PCA do not.